Forty Years of Amdahl’s Law

Forty years ago, Gene M. Amdahl published a landmark paper, and described what has come to be known as “Amdahl’s Law.” This law can be viewed as casting a negative light on the prospects for parallel computation; few applications can achieve linear speedup (in terms of the number of processors), and in most cases, speed improvements are asymptotic. The key insight from Amdahl is that in software applications, it is not uncommon for one computation to require the results of a prior stage. This serialization of tasks makes massive parallelism impractical. While one might gain some improvement, a factor of three or four might be the best one could hope for in many instances. This observation is not welcome news for the computing field. Due to power constraints, modern microprocessors are being forced into “multi-core” configurations. For the processors to be successful, however, there must be a commercial market for them – and history has shown that despite the best efforts of designers and software developers, the market for parallel computation has never grown outside of scientific and server applications. The original paper by Amdahl begins with the following: For over a decade, prophets have voiced the contention that the organization of a single computer has reached its limits and that truly significant advances can be made only by interconnection of a multiplicity of computers in such a manner as to permit cooperative solution. Amdahl was correct in his analysis; for forty years, those who have pursued faster single-processor performance have thrived, while those who advocate parallel computation have suffered. This talk will highlight research results from many authors who have studied parallel computation. The goal of the talk is to stimulate active discussion on what the right strategy is to deal with our current set of barriers to increased performance. Blindly following the lead of the Prophets of Parallelism is unwise; the track record for these leaders is remarkably poor.